Principled Parsing for Indentation-Sensitive Languages

Many languages, such as Haskell, Python, and F#, use the indentation and layout of code as part of their syntax. Because context-free grammars are not able to express these layout rules, existing parsers use ad hoc techniques to handle them. These techniques tend to be low-level and operational in nature, and thus forgo the advantages of more declarative specifications like context-free grammars. For example, they are often coded by hand instead of being generated by a parser generator. This paper presents a simple extension to context-free grammars for expressing these layout rules and derives CYK, GLR, and LR(k) algorithms for parsing these languages. These grammars are easy to write and can be parsed efficiently. Example for several languages are presented, as are benchmarks showing the practical efficiency of these algorithms.